1. Field of the Invention
The present invention relates to an image processing apparatus, an image processing method and an image processing program recording medium for inputting digital photographic image data containing dot-matrix pixels and converting each pixel of the image data according to predetermined correspondence relationship information, and it is also concerned with a color adjustment method, a color adjustment device and a color adjustment control program recording medium for carrying out optimum color adjustment.
2. Description of Related Art
There are a variety of conventional image processing techniques for processing digital photographic image data, more specifically such image processing techniques for contrast enhancement, chromaticity correction, and brightness correction, for example. Each pixel of image data is converted according to predetermined correspondence relationship information in these conventional image processing techniques. In an example of chromaticity correction, a color conversion table is prepared beforehand, and when original image data is input, the color conversion table is referenced to produce output image data. Thus, in flesh color correction, for example, a flesh color part of an image is made vivid on output.
In conventional image processing techniques mentioned above, however, when correction of certain chromaticity is made, an entire image may be affected by the correction, causing an unsatisfactory result of image adjustment. For instance, if it is attempted to make a pallid complexion of a person""s image look ruddy, an entire image may become reddish. Namely, although a certain kind of image adjustment provides a desired effect on an image, it may cause an undesired side effect thereon.
It is therefore an object of the present invention to provide an image processing apparatus which is capable of minimizing an adverse side effect in adjustment of an image data.
In accomplishing this object of the present invention and according to one aspect thereof, there is provided an image processing apparatus for inputting photographic image data containing dot-matrix pixels and converting each pixel of the image data according to predetermined correspondence relationship information, which comprises the correspondence relationship holding unit which holds a plurality of correspondence relationship information of the image data in combination with applicable object information thereof, the correspondence relationship judging unit which judges each correspondence relationship for converting each pixel of the image data according to the correspondence relationship and applicable object information which are held by the correspondence relationship holding unit, and the image data converting unit which converts each pixel of the image data by referencing each correspondence relationship judged by the correspondence relationship judging unit from the correspondence relationship holding unit.
Where an image contains dot-matrix pixels and each pixel is indicated in image data representation, each pixel of image data is converted according to predetermined correspondence relationship information in the above-mentioned arrangement of the present invention. In this image processing, a plurality of correspondence relationship information for image data conversion and applicable object information are held in combination. According to these information, a judgment is formed on each correspondence relationship for conversion of each pixel of image data, and based on result of judgment, conversion of each pixel of image data is carried out.
More specifically, there are provided plural correspondence relationships for image data conversion, and it is judged which correspondence relationship is to be applied to each pixel. Then, image data conversion is performed by applying a proper correspondence relationship to each pixel. For example, a correspondence relationship for making green color of tree leaves more vivid is applied to green color pixels indicating tree leaves, and a correspondence relationship for making a complexion ruddy is applied to flesh color pixels. In this manner, each correspondence relationship is flexibly applied as required.
In accordance with the present invention, since correspondence relationship information for image data conversion can be changed for each pixel, it is possible to provide an image processing method which allows execution of desired image processing without giving an adverse effect to unspecified parts of an image.
It will be obvious to those skilled in the art that the image processing method in which plural correspondence relationship information can be changed for each pixel is practicable not only in a substantial apparatus, but also in a system on its method. From this point of view, the present invention provides an image processing apparatus of inputting photographic image data containing dot-matrix pixels and converting each pixel of the image data according to predetermined correspondence relationship information, in which a plurality of correspondence relationship information for conversion of the image data and applicable object information thereof are held in combination, each correspondence relationship for converting each pixel of the image data is judged according to the correspondence relationship and applicable object information thus held, and each pixel of the image data is converted according to result of judgment.
The present invention for realizing image processing as mentioned above may be practiced in a variety of forms, i.e., it may be embodied in hardware, software or combination thereof in various arrangements that are modifiable as required. As an example of practicing the present invention, it may be embodied in image processing software. In this case, according to the present invention, there is provided a software storage medium for recording the image processing software. From this viewpoint, the present invention provides a storage medium for recording an image processing program for inputting photographic image data containing dot-matrix pixels on a computer and converting each pixel of the image data according to predetermined correspondence relationship information, the image processing program comprising the steps of; holding a plurality of correspondence relationship information for conversion of the image data in combination with applicable object information thereof, forming judgment on each correspondence relationship for converting each pixel of the image data according to the correspondence relationship and applicable object information thus held, and converting each pixel of the image data according to result of judgment thus formed. In this case, the same functionalities as in the foregoing description can be provided.
The storage medium for recording the image processing program may be a magnetic storage medium, a magneto-optical storage medium or any software storage medium to be developed in the future. Obviously, the present invention embodied in image processing software may take such a replicated form as a primary replicated product, secondary replicated product, etc. In addition, the image processing software according to the present invention may be supplied through use of a communication line or in a form of contents written in a semiconductor chip.
Still more, there may be provided such an arrangement that some parts of the present invention are embodied in software while the other parts thereof are embodied in hardware. In a modified embodiment of the present invention, some parts thereof may be formed as software recorded on a storage medium to be read into hardware as required.
In image processing according to the present invention, photographic image data is particularly suitable as object data to be processed. However, it is not necessarily required to use 100% photographic image data as object data, and semi-synthetic image data or composite-photographic image data produced from plural photographic images may be used as object data.
As stated above, in accordance with the present invention, there are provided a plurality of correspondence relationships in combination with applicable object information thereof. Each correspondence relationship is used to perform specific conversion of input image data, and for each correspondence relationship, there is provided applicable object information thereof. It is to be understood that such applicable object information may take a variety of forms.
Another object of the present invention is to provide an image processing method of using applicable object information.
According to the present invention, there is provided an image processing apparatus wherein, in the correspondence relationship information holding unit, each correspondence relationship applicable to each part of image is held along with information on each part applicable to each correspondence relationship, and wherein, in the correspondence relationship judgment means, a part of image corresponding to each pixel is detected and each correspondence relationship for conversion is judged through comparing the detected part of image with part information for each correspondence relationship.
In the above-mentioned arrangement of the present invention, a unique correspondence relationship is applicable to each part of image, and there is provided part information applicable to each correspondence relationship. Therefore, a part of image corresponding to each pixel is detected, and the detected part of image is compared with part information for each correspondence relationship. Thus, if a match is found, image data conversion is carried out using relevant correspondence relationship information.
More specifically, a unique correspondence relationship is provided for each part of image and it is applied to image data to be converted. For instance, a correspondence relationship for sky-blue color is applied to an upper part of image, while performing no conversion on a lower part of image. In this case, a part of image is not limited in its size and shape, and it may take any shape such as a rectangular shape, round shape, free shape, etc. Furthermore, it is not necessarily required to specify a part of image in a particular range. A part of image may be specified with respect to a certain center point. In this case, a degree of adjustment in image conversion may be decreased gradually as being apart from the center point. Obviously, a certain correspondence relationship may be applied to a plurality of parts of image.
As stated above, in accordance with the present invention, since each correspondence relationship is changed with each part of image, it is possible to carry out image processing with a particular location specified, thereby permitting easy operation.
Another object of the present invention is to provide an image processing method of specifying an applicable object.
According to the present invention, there is provided an image processing apparatus wherein, in the correspondence relationship information holding unit, a plurality of correspondence relationships are held along with information on chromaticity applicable to each correspondence relationship, and wherein, in the correspondence relationship judgment means, a value of chromaticity of each pixel is detected and each correspondence relationship for conversion is judged through comparing the detected value of chromaticity with information on chromaticity for each correspondence relationship.
In the above-mentioned arrangement of the present invention, a unique correspondence relationship is applicable to each level of chromaticity. Therefore, a value of chromaticity of each pixel is detected, and the detected value of chromaticity is compared with information on chromaticity for each correspondence relationship in judgment on each correspondence relationship for conversion. In this case, chromaticity is used for the purpose of judgment on applicability of each correspondence relationship, i.e., it is not necessarily required to adjust a level of chromaticity.
In accordance with the present invention, since an object can be selected using chromaticity, its embodiment is relatively easy even if pixels to be processed are located at a plurality of parts of image.
In chromaticity judgment, input image data may be used intactly, or a color space scheme is changed before judgment so that input image data is converted into data easy for judgment on chromaticity. Still more, it is not necessarily required to use a narrow definition of chromaticity. Chromaticity judgment may be made in a wider definition by using a level of any predetermined recognizable characteristic of image data.
It is possible to adopt various units for setting correspondence relationship information for image data conversion.
Another object of the present invention is to provide an image processing apparatus of using a color conversion table.
According to the present invention, there is provided an image processing method wherein, in the step of holding the correspondence relationship information, a color conversion table is used for storing information on correspondence relationship between pre-converted original image data and post-converted image data.
In the above-mentioned arrangement of the present invention, the color conversion table retains pre-converted image data and post-converted image data. By referencing the color conversion table using pre-converted image data, it is possible to attain post-converted image data, i.e., correspondence relationship information is stored in a form of table.
Besides, it is possible to hold correspondence relationship information in such forms as arithmetic expressions, arithmetic parameters, etc. However, in use of a table form, there is an advantage that operations can be simplified since just referencing is required.
As stated above, in accordance with the present invention, since correspondence relationship information is held in the color conversion table, it is relatively easy to carry out image data conversion.
As to a plurality of correspondence relationships, it is not necessarily required to provide different kinds of correspondence relationships.
Another object of the present invention is to provide an image processing apparatus of attaining results of plural conversions substantially.
According to the present invention, there is provided an image processing apparatus wherein, in the correspondence relationship information holding unit, a plurality of correspondence relationships are realized by changing a degree of correspondence relationship applicability.
In the above-mentioned arrangement of the present invention, a plurality correspondence relationships are set up by changing a degree of correspondence relationship applicability. For example, according to a degree of applicability, relational association can be made between image data not applied to a certain level of correspondence relationship and image data applied thereto. In this case, either linear or non-linear relational association may be made.
Still more, in use of such a degree of applicability, just a single correspondence relationship can set up a condition converted by applying the correspondence relationship and a condition not converted substantially, i.e., it is possible to provide a plurality of correspondence relationships. In this case, more correspondence relationships can be realized by changing a degree of applicability as required.
As stated above, in accordance with the present invention, it is possible to rearrange a single correspondence relationship as a plurality of correspondence relationships by changing a degree of applicability.
Another object of the present invention is to provide an image processing method of holding a plurality of correspondence relationships which may be predetermined or specified as required.
According to the present invention, there is provided an image processing apparatus wherein, the correspondence relationship information holding unit includes the correspondence relationship specifying unit in which a plurality of correspondence relationships applicable to the image data are specified in succession as required.
In the above-mentioned arrangement of present invention, a plurality of correspondence relationships to be applied to the image data are specified in succession. There are two kinds of correspondence relationship elements; a condition to be converted, and an object to be applied. Either one or both of these kinds of elements may be taken. In this case, it is also possible to specify a part of each image and select a conversion condition for the specified part of image. Still more, there may be provided such an arrangement that a flesh color adjustment item or a sky-blue color adjustment item is selectable as an optional function of image processing.
As stated above, in accordance with the present invention, since applicable correspondence relationships can be specified as required, it is possible to improve a degree of freedom in image processing.
Another object of the present invention is to provide an image processing apparatus of holding each correspondence relationship for adjusting brightness.
According to the present invention, there is provided an image processing apparatus wherein, in the correspondence relationship information holding unit, each correspondence relationship for adjusting brightness of the image data is held.
In the above-mentioned arrangement of the present invention, each correspondence relationship for adjusting brightness according to the image data is held, and brightness of predetermined pixels is adjusted using the correspondence relationship thus held. For example, on a certain part of image, brightness of pixels thereof is increased.
As stated above, in accordance with the present invention, it is possible to adjust brightness on a part of image based on applicable correspondence relationship information.
Another object of the present invention is to provide an image processing apparatus of holding each correspondence relationship for adjusting chromaticity.
According to the present invention, there is provided an image processing apparatus wherein, in the correspondence relationship information holding unit, each correspondence relationship for adjusting chromaticity according to the image data is held.
In the above-mentioned arrangement of the present invention, each correspondence relationship for adjusting chromaticity according to the image data is held, and chromaticity of predetermined pixels is adjusted using the correspondence relationship thus held. For example, on a part of image containing flesh color pixels, enhancement is made to provide ruddy flesh color.
As stated above, in accordance with the present invention, it is possible to adjust chromaticity on a part of image based on applicable correspondence relationship information.
Another object of the present invention is to provide an image processing apparatus of adjusting color vividness.
According to the present invention, there is provided an image processing apparatus wherein, in the correspondence relationship information holding unit, each correspondence relationship for adjusting color vividness according to the image data is held.
In the above-mentioned arrangement of the present invention, each correspondence relationship for adjusting color vividness according to the image data is held, and vividness of predetermined pixels is adjusted using the correspondence relationship thus held. For example, to enhance color of any subject on a surrounding background, color adjustment is made so that the color of the subject becomes more vivid.
As stated above, in accordance with the present invention, it is possible to adjust color vividness on a part of image based on applicable correspondence relationship information.
If it is distinguished clearly whether a predetermined correspondence relationship is to be applied or not, a conspicuous boundary may take place in an image. Therefore, another object of the present invention is to make such a boundary unrecognizable by providing an image processing apparatus for adjusting a level of correspondence relationship gradually.
According to the present invention, there is provided an image processing apparatus wherein, in the image data conversion unit, for pixels in a transition region between regions which are different in terms of correspondence relationship, a level of correspondence relationship is adjusted gradually.
In the above-mentioned arrangement of the present invention, a transition region is provided between regions which are different in terms of correspondence relationship, and for pixels in the transition region, a level of correspondence relationship is adjusted gradually in image data conversion, thereby eliminating a stepwise difference in a processed image.
As stated above, in accordance with the present invention, it is possible to suppress occurrence of a stepwise difference on a boundary between regions to which different correspondence relationships are applied respectively in an image.
A level of correspondence relationship to be adjusted gradually in a transition region is selectable as required, and either linear or non-linear transition may be applied.
As mentioned in the foregoing description, correspondence relationship information for image data conversion can be used for color adjustment. However, as to selection of an element color and a degree of enhancement, it is required for a human operator to make a decision. In color adjustment processing of digital image data, it is difficult to identify which part will cause conspicuous color deviation in advance.
For instance, in case of a color image of a building, color deviation will not be recognized as long as an original color thereof is unknown. However, in case of a flesh color image of a person""s face, an original color thereof can be presumed in most cases. Therefore, adjustment of flesh color in an image is performed by the operator so that natural flesh color is given.
In color adjustment, there is a problematic factor which is referred to as a memory color effect in psychology. When a lemon is photographed as a subject, for example, a color of a photographed lemon image looks slightly somber even if it meets a color of the actual lemon in measurement using a colorimeter. Agreement between the color of the lemon image and that of the actual lemon cannot be found unless a side-by-side comparison check is made. Due to a memory color effect in psychology, a person memorizes that a color of a lemon is vivid in general. However, an actual lemon does not have such a vivid color as is memorized. Therefore, even if a lemon image is corrected to have its actual color, it is not satisfactory in terms of visual sensation in psychology. In case of flesh color, green color of tree leaves and blue color of sky, a memory color effect tends to occur as stated above.
When a person conducts adjustment of flesh color, for example, a degree of flesh color enhancement is determined in an appropriate range though the flesh color does not exactly match its memory color. If the flesh color is adjusted to exactly match its memory color, deviation will occur in other colors.
Therefore, automatic color adjustment cannot be performed just by predetermining correspondence relationship information. Even in automatic color adjustment, it is rather difficult to attain satisfactory result due to a memory color effect, i.e., a person must readjust result of automatic color adjustment.
It is another object of the present invention to provide a color adjustment device, a color adjustment method and a storage medium for recording a color adjustment control program for carrying out optimum color adjustment processing automatically while taking account of such factors as a memory color effect.
According to the present invention, there is provided a color adjustment device for performing color separation of color image data for each predetermined element color and adjusting the color image data in enhancement to provide each desired color in result of color image output delivered for each element color by such a device as an image output device, etc., comprising; the chromaticity judging unit which determines a value of chromaticity of each pixel according to the color image data, the object chromaticity pixel statistical calculation unit which performs statistical calculation on pixels having chromaticity values which have been determined to meet a predefined range of chromaticity by the chromaticity judging unit, the color adjustment degree judging unit which determines a degree of color adjustment so as to eliminate a difference between a predetermined optimum value for pixels meeting the predefined range of chromaticity and a result value attained in the statistical calculation and for regulating the degree of color adjustment according to an occupancy ratio of pixels subjected to statistical calculation to the total number of pixels, and the color adjusting unit which carries out color adjustment of the image data according to the regulated degree of color adjustment.
In the above-mentioned arrangement of the present invention, a value of chromaticity of each pixel is determined according to the color image data. Since a value of chromaticity represents an absolute ratio in psychophysical color specification which does not depend on brightness, important parts (objects) in an image can be identified according to possible ranges of chromaticity. For example, an object can be distinguished by checking whether its chromaticity value is in a possible chromaticity range of flesh color or green color of tree leaves. Based on this principle of chromaticity, if determined chromaticity values of pixels are in a predefined chromaticity range, statistical calculation is performed on these pixels. In the above example, if determined chromaticity values of pixels are in the possible chromaticity range of flesh color, they are subjected to statistical calculation. For statistical calculation, an average value, median, etc. may be taken. Statistical calculation on pixels having chromaticity in a predefined range with respect to all the pixels of an image is analogous to a situation that a person judges an average level of flesh color with respect to flesh-color-like pixels.
Then, it is necessary to judge whether or not a particular color matches its memory color. A degree of color adjustment is so determined as to eliminate a difference between a predetermined optimum value for pixels meeting a predefined range of chromaticity and a result value attained in the statistical calculation. However, if a degree of color adjustment thus determined is applied intactly, a flesh color part will be adjusted to have memory flesh color, for example, but non-allowable color deviation will occur on other color parts. Therefore, the degree of color adjustment thus determined is regulated according to an occupancy ratio of pixels subjected to the statistical calculation to the total number of pixels. In this manner, color adjustment is made properly so that adjustment of flesh color will not cause an excessive change in other colors. Thus, the image data is color-adjusted according to the regulated degree of color adjustment.
As stated above, in accordance with the present invention, it is possible to provide a color adjustment method which enables automatic color adjustment processing through chromaticity statistical calculation for simulating human judgment.
It is to be understood that a method of performing statistical calculation on pixels having chromaticity in a predefined range and reflecting result of statistical calculation in a degree of color adjustment is practicable in a substantial apparatus for its implementation. From this point of view, the present invention provides a color adjustment device for performing color separation of color image data for each predetermined element color and adjusting the color image data in enhancement to provide each desired color in result of color image output delivered for each element color by such a device as an image output device, etc., comprising the steps of; determining a value of chromaticity of each pixel according to the color image data, performing statistical calculation on pixels having chromaticity values which have been determined to meet a predefined range of chromaticity, determining a degree of color adjustment so as to eliminate a difference between a predetermined optimum value for pixels meeting the predefined range of chromaticity and a result value attained in the statistical calculation, regulating the degree of color adjustment according to an occupancy ratio of pixels subjected to the statistical calculation to the total number of pixels, and carrying out color adjustment of the image data according to the regulated degree of color adjustment.
As an example of practicing the present invention, it may be embodied in color adjustment control software for a color adjustment device. In this case, according to the present invention, there is provided a software storage medium for recording the color adjustment control software. From this viewpoint, the present invention provides a storage medium for recording a color adjustment control program for performing color separation of color image data for each predetermined element color on a computer and adjusting the color image data in enhancement to provide each desired color in result of color image output delivered for each element color by such a device as an image output device, etc., the color adjustment control program comprising the steps of; determining a value of chromaticity of each pixel according to the color image data, performing statistical calculation on pixels having chromaticity values which have been determined to meet a predefined range of chromaticity, determining a degree of color adjustment so as to eliminate a difference between a predetermined optimum value for pixels meeting the predefined range of chromaticity and a result value attained in the statistical calculation, regulating the degree of color adjustment according to an occupancy ratio of pixels subjected to the statistical calculation to the total number of pixels, and carrying out color adjustment of the image data according to the regulated degree of color adjustment. In this case, the same functionalities as in the foregoing description can be provided.
In determining chromaticity of each pixel of color image data, chromaticity conversion is performed taking account of a color space coordinate scheme used for original color image data. In this case, chromaticity is used intactly if it is a direct element, or chromaticity is converted if it is not a direct element. For conversion, a conversion table or a conversion expression may be used. In this case, it is not necessarily required to attain an accurate value in result, i.e., an error maybe contained if its adverse effect is insignificant.
When determined chromaticity values of pixels are in a predefined chromaticity range, statistical calculation is performed on these pixels. In this case, it is not necessarily required to make an alternative-choice judgment, i.e., it is not necessarily required to determine whether the predefined chromaticity range is satisfied or not. Instead, statistical calculation may be made while changing a weight factor.
A variety of statistical calculation approaches may be used in application.
Another object of the present invention is to provide a color adjustment method which is advantageous in the amount of calculation.
According to the present invention, there is provided a color adjustment device wherein: in object chromaticity pixel statistical calculation unit, an average value of pixels judged to be object pixels is calculated for each element color of color image data; and in color adjustment degree judging unit, an optimum value of each element color is provided for color image data meeting the predefined range of chromaticity, and a degree of color adjustment is determined according to the optimum value of each element color.
In the above-mentioned arrangement of the present invention, statistical calculation is performed for each element color of color image data to determine an average value of pixels judged to be object pixels, i.e., statistical calculation is carried out using each element color of color image data instead of chromaticity statistical calculation. On the other hand, an optimum value of each element color is provided for color image data meeting the predefined range of chromaticity. In judgment of a degree of color adjustment, comparison for each element color is performed, and a difference attained is used as a degree of element color adjustment.
In accordance with the present invention, it is judged from chromaticity whether or not an objet is applicable to statistical calculation. Since statistical calculation is performed for each element color of color image data, statistical result can be used easily in subsequent arithmetic operations.
Besides, it is possible to use median calculation, standard deviation calculation, etc. though the amount of calculation increases in statistics.
A range of chromaticity to be subjected to statistical calculation can be changed as required.
Another object of the present invention is to provide a color adjustment device which is suitable for memory color adjustment.
According to the present invention, there is provided a color adjustment device wherein, in the object chromaticity pixel statistical calculation unit, statistical calculation is performed on object chromaticity pixels in a possible range of chromaticity in terms of memory color in psychology.
In the above-mentioned arrangement of the present invention, statistical calculation is performed on object chromaticity pixels which meet a possible range of chromaticity in terms of memory color in psychology. In this case, it is not necessarily required to attain statistical result using just one possible range of chromaticity. It is possible to carry out a plurality of statistical calculations respectively by specifying plural possible chromaticity ranges.
In accordance with the present invention, since calculation is performed on pixels corresponding to memory color, a degree of color adjustment is regulated according to the number of pixels, thereby making it possible to form a judgment similar to that by a person.
In judgment of a degree of color adjustment, a degree of color adjustment is so determined as to eliminate a difference between a predetermined optimum value for pixels meeting a predefined range of chromaticity and a result value attained in the statistical calculation, and the degree of color adjustment is regulated according to an occupancy ratio of pixels subjected to statistical calculation to the total number of pixels. In this case, it is not necessarily required to carry out these two steps of processing. It is just required to provide the same result of processing. For example, if a difference between a predetermined optimum value for pixels meeting a predefined range of chromaticity and a result value attained in statistical calculation is obtained in a step of statistical calculation, it may be reflected in result of statistical calculation. If an occupancy ratio of pixels subjected to statistical calculation to the total number of pixels is obtained in the step of statistical calculation, it may be reflected in result of statistical calculation. Sill more, it is not necessarily required to perform subtraction to attain a difference between a predetermined optimum value and a result value in statistical calculation. It is just required to include any proper calculation inherently.
A degree of color adjustment may also be determined in a variety of forms.
Another object of the present invention is to provide a color adjustment device of determining a degree of color adjustment.
According to the present invention, there is provided a color adjustment device wherein, in the color adjustment degree judging unit, a tone curve representing input-output relationship information is used for changing a degree of element color enhancement, and a tone curve is formed according to the degree of color adjustment.
In the above-mentioned arrangement of the present invention, a tone curve is formed according a degree of color adjustment, and a degree of element color enhancement is changed according to the tone curve thus formed so that input-output relationship information on each element color is represented by the tone curve.
In accordance with the present invention, since a tone curve is used, it is easy to add adjusted conditions of other element colors from an overall point of view. In this case, a memory color part is adjusted toward an ideal color level and adjacent color regions are adjusted gradually in a continuous manner, thereby preventing occurrence of pseudo-contouring due to tone jumping.
The above and other objects, features and advantages of the present invention will become more apparent from the following description of embodiments with reference to the accompanying drawings.